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_ Pfogram evaluation enables administrators to 

erits of existing progftams and the need for new ,ones. 
revision, deletion, or,inception of educational 
review examines general information on program 


evaluation, including evaluation design,. methodology, and 
bibliographic materials. Eleven of the documents reviewed are 
available from the ERIC Document Reproduction |Servi¢ge. A " 
fsupplementary bibliography lists ddcuments on program evaluation 
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\ # Effective school program evaluation isa dynamic, vital foun *, | 
‘ . i dation for instryctionwl improvement, Educators undertaking a 
ar a 3 school eValuation are taking inventory of all phases of the school ‘ 
hy > é F program. From the evaluation, changes in practice can be insti- A 
* ar: . 
= i i “5 > tuted or cohtinuatiey of existing practices can go on with-yome . 
ae a : assurance that a school program’of acceptable quality exists... . 
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nf * Evaluation does not imply that something defective exists, 4 : 
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7 + A . . ° su * 
r . practice will be nurtured and continued. Schod! evaluation * 
4 rm . 
% F \ t ,makes good schools even better. " * . 
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i Program evaluation enables administeators to watch over the. F 
‘ ‘i i = 3 educational.process. Evaluation can determine the merits of * 
\ a “6 ae as 
. Fy existing programs and the need for new ones. It ean lead to ' 
3 i Yi : a: revision, deletion, or inception of educational programs. . YJ iy 
WW +7 me The importance of program evaluation to the process of ; 
Z c a RN education has engendered a large body of research and specu-« 
- - To :4 ‘lation. The documents in this review concentrate, for the : 
bess 2 & \ most part, on methods of program evaluation. A few docu- a 
a ments of importance concerning general information and t 
evaluation design are included, as are s€veral bibliographic 
QS works. A supplementary bibliography, lists documents on, ‘ 


program evaluation within specific program areas. = 
Eleven of the documents reviewed) are available from the 
“ERIC Document Reproduction Seryice. Complete ordering 
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GENERAL INFORMATION 


A guide by the University of Kentucky 
(1964) maintains that conscientious evalua 
tion, including self-evaluation by educators, 
isgessential to program, improvement, The 
guide focuses on program goals and objec- 
tives, staff, administration, finance, school 
plants, and transportayon as’ they” relate 
to evaluation, © 

The Kentucky study identifies a problem 
in evaluation design. Som® Apects of the 
program are difficul( or ‘impossible | to, 
measure, The only alternative to direct 


Sah 
measurement is inference: 
‘ 


4 

We simply assume : without avery sound 
reason for doing so, except that we have no 
other alternative—that what is measurable 
(including much we honestly cannot meat 
sure very accurately) is correlated with some 
impOrtant things we do not know how to 
measure. 


Curriculum evaluat ion involves collecting, 
processing, and interpreting both objective 
and subjective data (Stake 1967). The objec- 
tive data include descriptions of goals, envi-. 
ronments, personnel, methods, content, and 
outcomes. Subjective data are personal judg- 
ments by the e aluatof 

Stake cites-the need for new techniques 
of observation and judgment with greater 
attention to diagnostic testing, task analysis, 
and goal evaluation. He Sbelieves ‘that’ be- 
havioral science techniques should be uti- 
lized in curriculum evaluation. 


EVALUATJON DESIGN 


Nivette (1969)¢cites three requirements 
of educational objectives. First, objectives” 
Khould describe what the student does. Sec- 
“ond, they should describe conditions,under 
«Which the student's performance cin be 
observed.-Third, objectives should describe 
the standards the student must meet. 
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/ communication and interaction with key 

_ dec ision-makers is a cornerstone on which 
effective evaluation rests.” 

Randall (1969) 


4 
. . 


1 . se . 4. 

The definition of educattonal objectives 
in is the ‘first’ step) in 
’ . . ‘ ‘ ° * 
Nivette’s evaluation design, The objectives 


behavioral terms 
must then be translated into descriptions of 
behavieos, and the evaliator must identify 
situations in which the designated pehavior 


<ean be observed. An interpretive device to 


measure desired growth is the next require: 
ment of An evaluation design. Finally, the 
eyaluator must state his conclusions regard- 


. ing the extent to which the objectives are 


achieved. ; +s 
Nivette also includes in his study alter- 
native methods for designing evaluation 


procedures, five evaluative designs, and a 


adiscussion of the use of statistics in evalua- 


tion design. a4 
A booklet by the Educational Innovators 


* PressA 1970) defines performance and proc- “5 


ess Gbjettives and ‘provides a set of pro-/ 
cedures *for developing and * writing each. 
Both types of objectives contribute to the 
integration of accountability into evaluation 
of an educatfonal program. ‘ 
both 
measures 


Unless norm- and critérion- 


referenced are used, realistic 
evaluations cannot Be, made (Klein 1972). 
Norm-referenced measures. compare per; 
formance tO a norm or“uverage, criterion- 
referenced) measures to a predetermined 
criterion. Klein, regards the essential dif- 
ference between the two types of measures 
asa question of interpretation, not measure- 
ment. He cites the need for both norm- and 
criterion-referenced interpretation in identi- 
fying program components needing improve: 
ment and students needing special attention, 


, \ 
in’ providing the basis for acQountability 
systems, and in determining wh\ther a pro: 
gram is beimg implemented as planed. 

An eviluation in? The North 
Central Association Quarterly (“Evaluation 
Guide». .” 


guide 


1969) characterizes:evaiuation 
as a foundation for educational change. 
The authors deseribe preparation, evalua 
tlon,s and follow-up procedures used by 
members of the associttion and note the 
ongoing naturelof evaluation: 

Evaluation is 4 continuous process, \s 


each decision -is made in the school about 


students, teachers, content, or services of 


fered, some evaluation is made concerning 


past performancesand future need. 
4 


Three, types of evaluation criteria are 


identified in the guide, Predictive criteria 
are quantitative measures such as teacher 
preparation details, number of volumes in 
the library, and length of ‘class period or + 
school day. Product criteria are behavioral 
changes in pupils. “Process criteria iwolve 
occturences during the ‘teaching process o1 


elsewhere in the selool ¢ xperience, 


EVALUATION MODELS 


According to Borg (1971), three evalua- 
tion models can be used to assess educational 
products and processes. In the taralidated 
form of experience model, the evaluator 
hypothesizes that certain specific pupil be- 
haviors will create changes in othe pupil 
behavier. He then develops a product or 
process designed 10. causenathe behaviors, 
cofects observational data, and makes an 
inferential leap to conclude success” or 
failure of the product or process. g 

The validated form of experience hes 
sadds .related research evidence tos the 
hypothgsis but is otherwise identic . to the 


first model. Phe direel pe rrormance evaltua- 


. 


e 


_ing relevant 


ana 
Progréin kealuation 


fon model cgmpares the performance of 

children e Xposed to the product or process 

to the per form, nce ofa “ ontol g Rroup.® I'he 

natureot the problem, available finds, con- 

straints such as tine, gnd the Consequences 

of mistakes in judgment will determine the 

-particubir evaluation model best suited: tq 
4 program. 

The EPIC (n.d.) de- 


Tines objec tive, evaluation of school pro- 


Evaltiation Center 


wams ats a process in which a school stall 
collects information to determine whether a 
given set of objectives has been met. The 
Evaluative Programs for Innovative Curricu- 
lums (EPIC) se heme distinguishes and de: 
fines three 1\ pe sof variables: instruétional, 
institutional, and behavioral. . 
The first step in the EPIC evaluation plan 
dnvelves identification of variables, composi 
tion of behavioral objectives, and develop- 
ment of the evaluative design. The second 
data 
gathering. Data analysis follows.: The final 


step Is program deseriplion sand 
step is the decision-making recycling proc- 
ess. To illustrate fis application, the entire 
EPIC procedure is applied to a spec ific 
program. 

A paper by Elartwig (1971 ) pre sents at de- 
libPately simplified evaluation model. The 


live steps inv oly edare ident Lifieation of need, 
desired) outcomes, 
\ment of a program, ongoing ev aluatlon, and 
terminal evaluation. : 
Randall (1969) describes the Stufflebeam- 
Guba context, input, process, and product 
(CIPP) model for evaluation of educational 
innovations. He sees evaluation as a combi- 
nation of effective 


statement of develop- 


based an 
timely} relevant: information. Problems. in 
evaluation include identifying decisions, and 


decisions 


decision-makers, timing decisions, identily- 


information, and 


reporting 


information in useful form. 


Randall identifies four classes of evalua- 
Gon: 
: a 7% : 

Context evaluation deals with the:set- 
- “ting of priorities aad the selection of 

strategies, . 

Design evaluation entails specification” 

of objectives and selection of the, 
2 ~ means to attain those objectives. 

Process evaluation assesses the possible 

need to restructure the program after 

the resultsof pilot testing and previous * 

evaluations arein. 

Product evaluation determines the ef- 

*fectiveness ofthe program in attaining 

its overall goals. \ . 

Roth (1971) maintains that formation 
obtained from — program 
usually subjective, vague, imprecise, Qd 
outdated. His five-step:program monitoring 
system stresses immediacy of feedback and , 
constant monitoring. The first step is state- 
ment of goals in specifig, measurable ‘terms. 
Next is the determination of valid behavioral 


evaluations — is 


. . . . . e f 
characteristics indicative of goal achieve- 


ment. Third;data afe obtained at established 
intervals and, fourth, collected and orga- 
nized at a central source. Finally, the pro- 
gram directors must review results regularly. 
_ The ~ Independent | Accomplishment 
Audit (IAA) measures student performance 
as a result: of financial outlays (Lessinger 
1970). The pre-audit: stage includes de- 
termination of objectives and establishment 
Of priorities. Translation involves the clear 
formulation of acceptable evidence. In the 
instrume nlation phase, instruments are 
determined and tec hniques established. The 
review calendar deteémines timing. The 
auditor*then carries out the audit and makes 
the results part of the public record. 


Fogel (1971), applies the techniques of, 


° 
‘ ‘ 


{« 


systemsanal¥sis to program evaluation, The, 
evaluator should determine the stated pro- 
gram goal and measure approximation to 
that goal. Then he must identily 
the program that might reduce the'discrep- 


glements of 


ancy between the stated goal “and -actual 
achievement. Elements outside the pregram 
ean also have an effect on goal achievement, 
and these clements should) be isolated. 
Individual processes and their* goals must 
be identified and analyzed and any addi- 


tional variables determined. From this sys; 


tems‘ analysis,, the evaluator should gain a,” 


fairly complete understanding of the pro- 
gram and a good idea of the weaknesses 
warranting further study. 

_ A handbook by Glassner’ (1969) provide s 
anor iéntation to the disere ‘paneyv 4 valuation 
model. This model positssthat the continu- 
ous reporting of observed differences be- 
tween the program staff's expec tations fora 


program and field reality will lead decision~ « 


makers to effect desirable changes. 

The first stage of the diycrepaney evalua- 
tion model stresses pepueeoacs of the 
evaluator and, the progrium staff in planning, 
generating, and 
evaluation design. The second stage com- 
pares the design to actual operation of the 
¢valuation. 

Rippey (1972) ‘argues for cooperation of 
both protagonists and antagonists of change 

- ona change-oriented team. His transactional 
evaluation model concentrates on the effects 
of change on those bringing about the 
change. _ + ‘ o° 
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BIBLIOGRAPHIC MATERIALS 


Eidell and ‘Klebe (1968) compiled an 
annotated bibliography of ‘sixty-four docu- 
ments on program evaluation. The docu- 
ments emphasize methods, procedures, and 


amending the program ~ 


ss 


” x P. . 


models, and deal ‘specifically with the 
evaluat n of subject’ progrants, state and 
federal programs, cutriculum planning, and 
innovation. Some ¢ntries treat the deyelop- 
ment and use of mathematic al models and 


cost-benefit analysis. 


A bibliography compiled — by Kidder 
(1971) includes tventy-cight abstracts ol 
documents on program evaluation. The cn: 
tries focus on evaluation methods: and on - 


the contributions of programs to student 


’ 


development. , 
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Evaluation is,a continuous process. As each decision is, made in, the school about 
studehts, teachers, content, or services offered, some evaluation ts made concern- 
ing past performance and future need. “Hvaluation Gruide. (1969) 
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Communication and_ interaction with key decision-makers, is a gornerstone on 


which effective evaluation rests. Randall (1969) 
> 


Clearinghouse Accession Number: EA 005 1-44 


Prior to publication, the manuscript was submitted to the American Association of School Administrators 
for critical review and determination of professional competence. This publication has met such standards, 


Points of view or opinions, however, do not necessarily represent the official view or opinions of the 


American Association of School \dministrators, 


